Confounded coefficients: Accurately comparing logit and probit coefficients across groups

The logit and probit models are critical parts of the management researcher's analytical arsenal. We often want to know if a covariate has the same effect for different groups, e.g., foreign and domestic firms. Unfortunately, many attempts to compare the effect of covariates across groups make the unwarranted assumption that each group has the same residual variation. If this is not the case, comparisons of coefficients can reveal differences where none exist and conceal differences that do exist. This article explains the statistical and substantive implications of this assumption, introduces approaches to comparing coefficients that avoid making it, and uses simulations to explore the practical significance of the assumption and the power of approaches introduced to avoid it. As a practical example, I show that an apparent dramatic new insight into the technology strategy of Japanese computer manufacturers is actually just a manifestation of this problem. I close with implications for the practice of research. Published: 2003 URL: http://www.business.uiuc.edu/Working_Papers/ CONFOUNDED COEFFICIENTS: EXTENDING RECENT ADVANCES IN THE ACCURATE COMPARISON OF LOGIT AND PROBIT COEFFICIENTS ACROSS GROUPS GLENN HOETKER College of Business University of Illinois at Urbana-Champaign 350 Wohlers Hall, 1206 S. Sixth Street Champaign, IL 61820 Phone: (217) 265-4081; Fax: (217) 244-7969; E-mail: [email protected] Version: October 22, 2004 Acknowledgements: I gratefully acknowledge the helpful comments of Paul Allison, Jongwook Kim, Tim Liao, and Steve Michael. All errors remain my own. Author’s Biography Glenn Hoetker is an assistant professor of strategy in the College of Business, University of Illinois at Urbana-Champaign. His research interests include social network analysis, the impact of national institutions on inter-firm relationships, and statistical methods. Recent publications include “Same rules, different games: variation in the outcomes of ‘Japanese-style’ supply relationships” (Advances in International Management, forthcoming) and “How much you know versus how well I know you: Selecting a supplier for a technically innovative component” (Strategic Management Journal, forthcoming). CONFOUNDED COEFFICIENTS: EXTENDING RECENT ADVANCES IN THE ACCURATE COMPARISON OF LOGIT AND PROBIT COEFFICIENTS ACROSS GROUPS ABSTRACT The logit and probit models are critical parts of the sociologist’s analytical arsenal. We often want to know if a covariate has the same effect for different groups, e.g., men and women. Unfortunately, many attempts to compare the effect of covariates across groups make the unwarranted assumption that each group has the same residual variation. If this assumption is false, comparisons of coefficients can reveal differences where none exist and conceal differences that do exist. Recent work has emphasized the theoretical potential for this problem and proposed a test of whether the effect of covariates differs across groups that is accurate, if limited, despite differences in residual variation. This paper extends these advances in three ways. First, it uses simulations to show that this theoretical problem is substantively significant under a wide range of common conditions, meaning that traditionally executed comparisons of logit coefficients should be viewed skeptically. Second, it uses simulations to assess the power of the test recently proposed to overcome the problem, finding that they are an improvement over naïve comparisons of coefficients, but have significant limitations. Third, it proposes and tests two alternative means of comparing coefficients across groups that avoid the assumption of equal residual variation entirely. The article closes with implications for the practice of research.